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(54) Detecting endoscope Insertion direction 

(57) A method of detecting the direction of insertion of an endoscope sheath (2) comprises the steps of forming from the 
same endoscope image a sequence of picture-Images differing In the number of pixels (or picture elements) and selectively 
locating a dark region (1, Fig. 12D) in a picture-Image of a predetermined number of pixels by successively Inspecting the 
picture-Images In the order of those having less pixels, and determining relative to thresholds, the grey levels of the 
respective pixels in the plurality of said picture-images thus formed, the above-mentioned dark region selected by the 
above-mentioned selective location step being considered to be the endoscope insertion direction. Preferably, the forming 
step includes means gradually forming a succession of picture-images of fewer pixels, reducing the number of pixels by a 
quarter in each step, so that. In the case where the number of pixels Is reduced to one quarter, the grey level of one 
macro-pixel In the picture-image of fewer pixels wiP be of an average value of the grey levels of the four subsidiary pixels in 
the 2 x 2 square region in the picture-image of more pixels corresponding to this one macro-pixel. 
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The claims were Sed later than the filing date within ftie period prescribed by Rule 25(1 ) of the Patents Rules 1982. 
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METHODS OF DETECTING ENDOSCOPE INSERTION DIRECTION: 

This Invention relates to methods of detecting the direction 
of insertion of an endoscope, particularly to methods adapted to 
facilitate the insertion of an endoscope In the inspection field of a 
5 large Intestine, possibly by automatic control. 

Recently, there has been increasing use of endoscopes to 
enable organs within a body cavity to be observed by inserting an 
elongated Insertion sheath* Various curative treatments can be 
applied by using medical tools inserted through a treating tool 
10 channel in the sheath as required. 

In a conventional endoscope Inspection, the doctor judges 
the advancing direction of the endoscope Insertion sheath by 
observing the endoscope image while inserting the sheath. 

However, a high degree of technical skill is required to 
15 insert an endoscope sheath when inspecting the large intestine. 

One object of the present invention is to provide a method 
of detecting the direction of Insertion of the sheath of an endoscope 
which enables the insertion direction to be detected automatically 
during the treatment. 
20 In accordance with the present invention a method of 

detecting the direction of insertion of an endoscope sheath comprises 
the steps of: 

forming an endoscope display Image, and providing means to 
construct therefrom a plurality of picture-Images differing in the 
25 number of pixels (or picture elements); 



selectively locating a dark region In a picture-image of a 
predetermined number of pixels by inspecting the picture-images 
sequentially in the order of less pixels to establish the grey levels 
of the respective pixels in the plurality of picture-images thus 
5 formed; and 

finally utilising the Baia dark region located by the said 
selective step as the endoscope sheath Insertion direction at that 
given instant. 

Preferably, the forming step includes gradually forming 
10 picture -images of fewer pixels, in each of which the number of pixels 
is reduced by a quarter so that, for each macro-pixel formed by four 
subsidiary pixels of the previous picture-image, the grey level of 
one macro-pixel in the nevly formed picture-Image of fewer pixels 
will have an average value of the grey levels of the four subsidiary 
15 pixels arranged 2 x 2 In the previous picture-image of more pixels 
that have been combined to form said one macro-pixel. 

Advantageously, the selective step includes inspecting the 
grey levels of the respective pixels in a picture-image, selecting 
that macro-pixel nearest to a required grey level, then inspecting 
20 the grey levels of the four subsidalry pixels of that selected 
macro-pixel and selecting the subsidiary pixel nearest to the 
required grey level. 

Other features of the present invention and their advantages 
will become apparent from the following description with reference to 
25 the drawings, in which:- 

Figure 1 is an explanatory view showing an endoscope with 
its insertion sheath inserted in the large intestine of a patient; 

Figure 2 is a perspective view showing a tip of an endoscope 
insertion sheath; 
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Figure 3 is an explanatory detail view showing the tip of an 
endoscope insertion sheath at a bent part of the large intestine ; 

Figure 4 is an explanatory view showing the type of 
endoscope display image which would be seen with the endoscope sheath 
5 tip as shown in Figure 3; 

Figure 5 is an explantory detail view showing the tip of an 
endoscope sheath inserted in a straight part of the large Intestine 
of a patient; 

Figure 6 is an explanatory view showing the type of 
10 endoscope image which would be seen with the endoscope sheath tip as 
shown in Figure 5; 

Figure 7 is an explanatory schematic view showing details of 
one exemplary embodiment of an endoscope apparatus using a 
fibre-scope and externally fitted television camera to operate in 
15 accordance with the present invention; 

Figure 8 is an explanatory schematic view showing details of 
one exemplary embodiment of an endoscope apparatus using a 
video-scope to operate in accordance with the present invention; 

Figure 9 is a partial flow chart showing Step 1 of one 
20 proposed method in accordance with the present Invention; 

Figures 10(A) to 10(D) are explanatory views showing . 
picture-images or pixels as arranged on respective "floors" of a 
typical "quad-tree", the sequence being restricted to simplify the 
illustration; 

25 Figure 11 Is an explanatory view showing the node 

relationships of the respective "floors" of the simplified 
"quad-tree" shown in Figure 10; 
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Figures 12{A} to 12(D) are explanatory views showing 
examples of picture-images of the respective "floors" of the 
"quad-tree" of Figure 10 with typical concrete numerical grey level 
values indicated; 
5 Figures 13(A) to 13(D) are explanatory views showing 

picture-images of respective "floors" of a typical "quad-tree" in a 
case where the endoscope image-display is octagonal instead of 
rectangular; 

Figure 14 is a partial flow chart showing Steps 2 to 27 of 
10 the proposed method referred to in Figure 9; 

Figure 15 is an explanatory theoretical view showing related 
"floors" of a "quad-tree" for explaining Step S22 of the Step 2 
described in flow chart from Figure 14; 

Figures 16(A) to 16(C) are explanatory views for explaining 
15 Step S24 of the Step 2 from Figure 14; 

Figures 17(A) to 17(D) are explantory views showing other 
examples of picture-images of the respective "floors" of the 
"quad-tree" in which concrete numerical grey level values are 
indicated; 

20 Figure 18 is a partial flow chart showing Step 3, together 

with the associated Steps 31 to 34 of the exemplary method, referred 

to in Figures 9, 14 and 15; 

Figure 19 is a partial flow chart explaining details of 

Steps 23 to 26 of Step 2 from Figure 14; and 
25 Figure 20 is an explanatory detail view showing a part of an 

endoscope apparatus as shown in Figure 7 or Figure 8, provided with a 

plurality of electronic computers. 
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First of all, a summary of the present Invention will be 
given by way of explanation vith reference to Figures 1 to 6. 

As shown In Figure 1, an endoscope 1 (of the type sometimes 
referred to as a fibre-scope) is provided vith an elongate 
5 flexible insertion sheath 2 connected to extend from a thick 

operating part 3. A flexible optical cable 4 extends from a side 
port of the operating part 3, and is provided at Its end vith a 
connector 5, which can be connected to a light output port of an 
illuminating source 6 containing a lamp 6a, The operating part 2 is 
10 provided at its rear end with an eye-piece fi. 

As shown in Figure 2, a rigid tip 11 is supported on a 
curvable section 12 provided at the remote end of the insertion 
sheath 2. The operating part 3 is provided vith a curvature-control 
operating knob (not illustrated) so that the curvable section 12 may 
15 be curved vertically and/or horizontally as required, by operation of 
this curvature-control operating knob. 

An illuminating lens 15 of an illuminating optical system, 
and an objective lens 16 of an observation optical system are 
arranged in the sheath end-face, directed substantially in the same 
20 direction as the axis of the rigid tip 11. A light guide (not 

illustrated), for example an optical fibre bundle, Is provided In the 
sheath 2 at the rear of the illuminating lens 15. This light guide 
is inserted through the insertion sheath 2, operating part 3, and , 
optical cable 4, and is connected to the connector 5 so that, when 
25 this connector 5 is coupled to the light source 6, the illuminating 
light emitted by the lamp 6a enters the light guide, to be led to the 
tip 11 and emitted from the tip end-face to be radiated out to an 
object through the illuminating lens 15, as indicated in Figure 2 by 
solid lines, 17 representing the illuminated region. 
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The end-face of an image-relaying fibre bundle (not 
illustrated) is arranged in the image-forming plane of the objective 
lens 16. This image guide is inserted through the insertion sheath 2 
and extends to the eye-piece 8. The object image formed by the 
5 objective lens 16 will be led to the eye-piece 8 so that it can be 
observed through the eye-piece lens mounted in the eye-piece 8- In 
Figure 2, a pair of broken-lines Indicate the visual field of the 
observing optical system. 

Row, as shown in Figure 2, the illuminating optical system 

10 and observing optical system of the endoscope 1 are adjacent to each 
other, and are directed in substantially the same direction. In the 
case. where there is no other illumination, the darkest part of each 
endoscope image will be that farthest from the tip. Therefore, as 
shown In Figure 1, in the case where the endoscope 1 has its 

15 insertion sheath 2 inserted into a closed tubular member, such as the 
large intestine 20 of a patient, the endoscope sheath may be inserted 
further in the direction of the darkest region of the obtained 
endoscope image. This will be explained with reference to Figures 3 
to 6, in which Figures 3 and A show one situation and its resultant 

20 image, and Figures 5 and 6 show the physical arrangement of another 
situation, together with the resulting Image, and in Figures 4 and 6 
zones of equal brightness are represented by similar shading, and the 
zones 21,22 and 23 show the regions in increasing order of 
brightness. 

25 Figure 3 shows the insertion sheath 2 of the endoscope 1 

inserted in the large intestine 20 of a patient, where it approaches 
an upward bend considered in the plane of the drawing. In such a 
case, as shown in Figure 4, the darkest part of the endoscope image 
will appear near the top of the Image. Therefore, In this case, the 



tip 11 of the endoscope 1 may be moved upward by controlled curvature 
of the curvable section 12 as the insertion sheath is advanced, so 
that the insertion sheath is guided upwards. 

Figure 5 shows the Insertion sheath 2 of the endoscope 1 
5 inserted in a straight part of the large intestine 20. In such a 
case, as shown in Figure 6, the darkest part in the endoscope image 
will appear at the centre- Therefore, in this case, the insertion 
sheath 2 of the endoscope 1 may be advanced further, with the 
curvable section straight, as it is. 
10 Thus, the method of detecting the required endoscope 

inserting direction is achieved by detecting the darkest region of 
the endoscope image, and accurately locating it. 

Exemplary embodiments of the present invention will now be 
described with reference to Figures 7 to 20. 
15 The proposed method can be applied, for example, using 

either of the embodiments shown in Figure 7 and Figure 8. 

The endoscope apparatus shown in Figure 7 is provided with a 
fibre-scope 1 that is fed with an illuminating light from a light 
source 6 via a guide 4, and an externally fitted television camera 30 
20 is coupled to the eye-piece 8 of the fibre-scope 1, which is 

generally as described with reference to Figure 1. The operating 
part 3 has an externally fitted television camera 30 provided, for 
example, with an image-forming lens (not illustrated) forming an 
image of light fed from the eye-piece 8, and a solid state imaging 
25 device (not illustrated) is arranged in the image-forming plane of 

this Image-forming lens. This externally fitted television camera 30 
drives the associated solid state imaging device, and is connected to 
a signal processing apparatus which forms the output signal cf this 



solid state imaging device into a standard video signal, the video 
signal output of the signal processing apparatus 31 is fed into a 
display monitor 32, and is also converted into digital form by an A/D 
converter 33 to be fed into an electronic computer 35, and possibly 
5 also into a memory (not illustrated) which may be within this 

electronic computer 35. The endoscope image will be displayed in the 
monitor 32 and the required insertion direction will be determined by 
the electronic computer 35. 

The alternative endoscope apparatus shown in Figure 8 is 

10 provided with a video-scope 41 in place of the operating part 3 and 
its externally fitted television camera 30, and this video-scope 41 
is coupled to the elongate flexible insertion sheath 2. A flexible 
optical guide 42 extends from a port in a side of the operating 
part 3, and is provided at its end with a connector 43 to connect to 

15 a control apparatus 45 containing a light source 6 and a video signal 
processing circuit 4S. A solid state imaging device (not 
illustrated) is arranged in the image-forming plane of the objective 
lens at the tip of the insertion sheath 2 and is connected to the 
video signal processing circuit 40 within the control apparatus 45 

20 through the signal lines inserted through the insertion sheath, 
operating part 3, optical guide 42 and the connector 43. The 
illuminating optical system of the video-scope 41 is the same as that 
of the fibre-scope 1, in that the illuminating light emitted from the 
lamp Sa of the light source 6 in the control apparatus 45 enters the 

25 light guide port. The solid state imaging device will be driven by 
the video signal processing circuit 46, and the output signal of this 
solid state imaging device will be processed to form a video signal 
by the video signal processing circuit. As in the endoscope using a 
fibre-scope, the video signal output from this video signal 



processing circuit 46 is fed to the monitor 32 for display, and also 
converted to a digital signal by the A/D converter 33, to be fed into 
the electronic computer 35, and possibly taken into a memory (not 
illustrated) within this electronic computer 35. The endoscope image 
5 will be displayed by the monitor 32 and the required insertion 
direction determined in this embodiment by the electronic 
computer 35. 

The proposed method is based upon the fact that the number 
of pixels in a digital form of the image is usually 512 x 512 pixels, 

10 with about 256 brightness gradations. However, in the following 
explanation, in order to simplify the description, the number of 
pixels in the original picture taken into the electronic computer 35 
will be assumed to be 8 x 8, with 8 grey level gradations of 0 black 
gradation and 7 white gradations. 

15 The proposed method in this first described embodiment 

comprises a first Step of forming, from an original picture fed into 

the electronic computer 35, a plurality of picture-images differing 

« 

in the number of pixels - Step 1. This is followed by a Step 2 for 
selectively locating any dark region in each picture-image of a 

20 predetermined number of pixels, by successively inspecting the grey 
levels of the respective pixels in the plurality of picture-inpges 
formed by the above-mentioned Step 1 in the order of the picture- 
images of less pixels, and then a Step 3 combining the region within 
the required grey level range near the region obtained in Step 2 with 

25 the adjacent regions obtained in Step 2. 

Step 1 will now be explained in detail with reference to 
Figures 9 to 11, and the analysis of individual pixel brightness 
levels, which will be referred to hereinafter as "grey levels". 



As shown in Figure 9, in the case where the total pixels of 
the orignal picture number n x n in Step Sll, an average of the grey 
levels of each region of 2 x 2 pixels is determined, and a picture- 
image of n/2 x n/2 pixels is obtained. Then, in Step 512, it is 
5 judged whether the number of pixels of the picture-image obtained in 
the Step Sll is or is not of the value 1.. In the case where the 
number of pixels is not 1, the Step Sll will be repeatedly carried 
out on the obtained picture-images until the case is reached where 
the number of pixels is 1, and Step 1 will then end and the 
10 inspection will proceed to Step 2. 

Thus, in Step 1, a succession of picture-images, each of 
less pixels, are formed until the number of pixels becomes 1, to make 
a so-called "goad-tree", (see Figure 11). 

Step 1 is further explained with reference to Figures 10(A) 
15 to 10(D), where xij represents a pixel coordinate, (wherein x ~ a,b,c 
or d) and <xij> represents a grey level of the pixel xij. 

As shown in Figure 10(D), in a Step Sll an average of grey 
levels of a region of 2 x 2 pixels is determined by a computation of: 

<cll> = 1/4 (<dll> + <dl2> + <d21> + <d22>) 
20 from an original picture (called a d-plane) assumed for simplicity to 
be formed of 8 x 8 pixels, dll,dl2 up to d88, and, as shown in 
Figure 10(C), a picture-image (called a c-plane) of 4 x 4 pixels, 
cll,cl2 up to c44, is obtained. In the same manner, as shown in 
Figure 10(B), a picture-image (called a b-plane) of 2 x 2 pixels, 
25 bll,bl2,b21 and b22 is then obtained from: 

<bll> = 1/4 (<cll> + <cl2> + <c21> + <c22>). 
Also, as shown in Figure 10(A), a final picture-image (called an 
a-plane) of 1 pixel, all, is obtained from: 

<all> = 1/4 (<bll> * <b!2> + <b21> + <b22>). 
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The above operations are nothing but forming picture-images 
of a resolution of 1/2 in turn. 

In the thus formed "quad-tree as shown theoretically in 
Figure 11, the nodes of the respective planes of a,b,c and d are 
related through pointers or links. 

Particularly, the node of the a-plane Is called a root node, 
and the node of the terminal (d-plane In this case) Is called a leaf 
node. 

Examples containing concrete numerical values of grey levels 
of respective pixels of respective picture-images obtained by the 
Step 1 are shown In Figures 12(A) to 12(D). 

The endoscope image display area is not always square, but 
is often octagonal or circular, due to the light distribution 
characteristics of the Illuminating system. Thus, in a case where it 
Is octagonal or circular, there will be no endoscope image, but 
always black regions in the four corners of the picture-Image. The 
process in such a case will now be explained for an octagonal display 
area, with reference to Figures 13(A) to 13(D). 

As shown in Figure 13(D), for example, the left upper corner 
on the d-plane has an entire pixel dll, and halves of the pixels dl2 
and d21 that are always black regions. In such a case, if <c\l> is 
computed by: 

<cll> = 1/4 (<dll> + <dl2> + <d21> * <d22>), 
<cll> will not show a correct value. In this case, if it Is computed 
by; 

<cll> = 1/3 {2(<dl2> + <d21> + <d22>)}, 
a correct value will be obtained. 



A genuine dark region so rarely exists in the extreme corner 
that the pixels dll,d!2 and d21 may be computed by: 

<cll> = 1/4 {7 + 7 + 7 f <d22>} 
with the white represented by the grey level 7. 
5 The case for an octagonal display area has been explained in 

detail, but the case for a circular or any other shape can be 
considered in exactly the same way. 

In the further explanations, in order to simplify the 
description, the endoscope image display area will always be 
10 considered to form a square. 

Step 2 will now be explained with reference to Figures 14 

to 17. 

As shown in Figure 14, first of all, in Step S21, a node of 
the value nearest to that of the required grey level is extracted 

15 from the picture-image of 2 x 2 pixels. To avoid confusion with the 
explanation of Step 1, in this section we will refer to the image 
planes of Step 1 as "floors", and the four lower node pixels formed 
below a pixel in one node will be termed "sons". Then, in Step S22, 
it is judged whether the inspection of the "floor" on which the 

20 above-mentioned Step S21 has been made may be concluded or not. In 
the case where the inspection of the "floor" may be concluded, in 
Step S23, it is judged whether the grey level of the node extracted 
in the above-mentioned Step S21 is in the required grey level range- 
or not. In the case where it is in the range, then in Step S24 it is 

25 judged whether the differences between the grey level of the node 
extracted in Step S21 and the grey levels of the four "sons" of the 
node are within a constant value k or not. In a case where they are 
within the constant value k, the node is registered as the required 
region. Step 2 ends, and the inspection proceeds to Step 3. 



On the other hand, Id Step S22, In a case where it Is judged that the 
inspection of the "floor" may not be concluded and, in Step S23, in a 
case where it is judged that the grey level is not in the required 
grey level range, then Step S21 and the later Steps are carried out 
5 again using the next "tree-level", that is, in the picture-image of 
the next largest number of pixels (the next lower "floor" in the 
"quad-tree"). In such a cas* r in the Step S21, a node of the value 
nearest to that of the required grey level is extracted from among 
the four "son" nodes corresponding to the node extracted by Step S21 

10 in the upper "floor" . Also, in Step S24, in the case where it is 
judged that the grey level is not within the constant value k, then 
in Step S26, it is judged whether the inspection has reached the 
lowest level of the tree to be searched, or not. In this Step S26, 
in case it is judged that the inspection may not be concluded, then, 

15 as described above, the Step S21 and the later Steps are carried out 
in the next "tree-level". Also, in Step S26, in a case where it is 
judged that the lowest level in the tree has been reached, then in 
Step S27 the inspection returns to the next higher "floor" (called a 
back-track) and the Step S21 and the later Steps are carried out. in 

20 the order from the node of the value nearest to that of the required 
grey level from among the remaining three "son" nodes belonging to 
the same parent node as the node on which, for example, the Step S21 
has already been made. If all four nodes have been tried, then the- 
inspection continues from the next higher level, and so on, 

25 The "floor" of which the Inspection in the Step S22 may be 

concluded shall be explained with reference to Figure 15. 

As already described, generally the number of pixel? is 
about 512 x 512. The structure of the "quad-tree* in such a case is 
shown in Figure 15. In Step 2, the inspection is started from the 



"floor" of 2 x 2 pixels and is advanced downward (to the "floors 11 of 
more pixels) but need not always be advanced only to the "floor" 
having no problem in the precision. This reduces the processing time 
by an electronic con^uter and can eliminate the influence of noise . 
5 On the other hand/ for example, the "floor" of 2 x 2 pixels is too 
coarse- Therefore, in Step S22, the upper limit of the "floor" of 
which the inspection may be concluded is set in response to the 
required precision. In Step S26, the lowest "floor" to be searched 
is selected. 

10 The Step S24 will now be explained with reference to 

Figures 16(A) to 16(C). 

As shown in Figure 16(A), if the differences between the 

grey level of the hatched part of a "floor" p and the grey levels of 

four "son" pixels corresponding to the above-mentioned hatched part 
15 in the lower "floor" q are within a constant value, the brightness 

will be so uniform that there need be no further inspection of the 

lower "floor" r. 

Therefore, in Step S24, when such a uniform state is found, 

no further inspection of the lower "floor" will be made. This also 
20 reduces the processing time by the electronic computer. 

In Step S24, in case it is Judged that the difference of the 

grey levels Is not within a constant value k and then, in Step S26, 

In the case where it Is judged that the inspection may be concluded, 

the non-uniformity of the brightness may be caused by a break In the 
25 optical fibres forming the image guide, or by noise, or 

non-uniformity in the Image. In such case, the error can be 

prevented by back -tracking in the Step S27, 
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In this embodiment, in Step S24 it is judged whether the 
differences between the grey level of the extracted node and the 
respective grey levels of four "son" nodes of that node are within a 
constant value Jc or not. An alternate method is shown In Figure 19. 
In Step S24, it Is judged whether the ratios of the grey levels of 
the respective nodes are within a constant value or not. In case the 
ratios are within the constant value, then Step 2 may end, but in a 
case where the ratios are not within the constant value, the process 

may proceed to the Step S2S. 

The operation of such Step 2 as in the above will be 

explained in the following by using two kinds of concrete numerical 

value examples. 

In the first example, the grey level of the required region 

shall be not more than 2. The grey level distribution shown in 

Figure 12 will be explained by way of example. The "floor" which may 

be concluded will be the c-plane. The constant value k of the 

Step S24 shall be 1.5. 

First of all, in the Step S21, the grey levels of four nodes 

on the b-plane are inspected. The node of the value nearest to that 

of the required grey level (not more than 2) is b21, which is the 

selected node. 

Then, in the Step S22, the b-plane is judged not to be a 
"floor" which may be concluded. 

Then, proceeding to the next "tree-level", in the Step S21, 
a node c32 of the value nearest to that of the required grey level is 
extracted from among the four nodes, c31,c32,c41 and c42, in the 
c-plane, corresponding to the selected note b21. 



Then, in the Step S22, the c-plane is judged to be a "floor" 
which may be concluded. 

Then, in the Step S23, the grey level of the node c32 is 
judged to be in the required grey level (not more than 2), 
5 Then, in the Step S24, the grey level of the node c32 is 

compared with the grey levels of the nodes d53,d54,d$3 and d64 in the 
d-plane corresponding to this node c32. The respective grey levels 
of c32,d53,d54,d63 and d64 are 1 to 2 and the difference is 1 and is 
judged to be within a constant value fc » 1.5. 
10 In the Step S25, the selected node c32 is then registered 

and Step 2 ends* 

In the second example, the grey level of the required region 
shall be not ©ore than 2 and the grey level distributions shown in 
Figures 17(A) to 17(D) shall be explained for example. The •floor" 
15 which may be concluded shall be the c-plane. The constant value k of 
Step S24 shall be 1.5. 

First of all, in Step 21, the grey levels of four nodes on 
the b-plane are inspected. The node of the value nearest to that of 
the required grey level (not more than 2) is b21, which is selected. 
20 Then, in Step S22, the b-plane is judged to be a "floor" 

which may be concluded. 

Then, the inspection proceeds to the next "tree-level" and, 
in Step S21, the node c32 of the value nearest to that of the 
required grey level is selected from among the four nodes of c31,c32, 
25 c41 and c42 on the c-plane corresponding to the selected node b21. 

Then, in Step S22, the c-plane is Judged to be a "floor" 
which may be concluded* 

Then, in Step S23, the grey level of the selected node c32 
is Judged to be in the required grey level (not more than 2). 
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Then, In Step S24, the grey level of the node c32 Id 
compared with the grey levels of d53, 654,463 and d64 on the d-plane 
corresponding to this node c32. The respective grey levels 
of c32,d53,d54,d63 and d64 are 0 to 2.5 and the difference is 2.5 and 
5 is judged not to be within a constant value k * 1.5. 

Then, in Step S26, it is judged whether the "floor" may be 
concluded at the bottom or not. As the bottom "floor" may be 
concluded, it is then back-tracked to the Step S27. That is to say, 
as the node c32 is not in the required grey level range, the same 
10 inspections in and after Step S21 are made in the order from the node 
of the value nearest to that of the required grey level among the 
other three nodes c31,c41 and c42 (in the order of c42,c41 and c31 in 
this case), the node c42 is selected and is registered in the 
Step S25 and Step 2 ends. 
15 . Step 3 will now be explained in detail with reference to 

Figure 18. 

First of all f in Step S31, the grey level of the nodes 
adjacent to the region obtained in Step 2 is inspected. Then, in a 
Step S32, It is judged whether there is a region within the required 

20 grey level range or not. In the case where there is such a region, 
in a Step 533, this region is combined with the region already 
obtained in Step 2 and Step S31 returns* The above Steps S31 to 533 
are continued until there is no adjacent node within the required • 
grey level range. On the other hand, in the Step S32, in the case 

25 where it is judged that there is no region within the required grey- 
level range, in a Step S34, it is judged whether the inspection of a 
higher precision Is required or not and, in case It is judged not to 
be required, Step 3 ends. On the other hand, in Step S34, in the 
case where it is judged that the inspection of a higher precision is 



required, the inspection proceeds to the next "tree-level", that is, 
to the lover "floor" and the inspection is made near the Steps S31 
to S33. 

Now the operation of this Step 3 will be explained, by way 
5 of example, with the concrete numerical values shown in Figure 12. 

In the Inspection in Step 2, the node c32 is extracted and 
therefore, in Step 3, the inspection In the vicinity is made. 

First of all, in the Step S31, the grey levels of the nodes 
of C21,c22,c23,c31,c33,c41,c42 and c43 near the node c32 are 
10 inspected. Then, in the Step S32, it is rJudged whether there Is a 
region within the required grey level range (not more than 2) or 
not. In this example, there is no region within the reguired grey 
level range. Then, in the Step S34, In case the inspection of a high 
precision is required, the d-plane Is more minutely inspected in the 
15 Steps S31 to S33. In this example, there is no region within the 
required grey level range. 

Therefore, in this example, there is no region to be 
combined with the region obtained in Step 2. 

As shown in Figure 20, if a plurality of electronic 
20 computers 35 are provided and a parallel process by the plurality of 
electronic computers Is made In the Inspection In the Steps 2 «and 3, 
the processing time will be able to be reduced. It Is one of the 
advantages of this embodiment using the "quad-tree" that the parallel 
process by the plurality of electronic computers Is possible. 
25 Thus, according to this embodiment, when the dark region 

selected by Steps 1 and 2 and possibly further combined by Step 3 in 
some cases, an endoscope insertion direction has been found. 
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Now, In the case where an original picture Is formed 
of n x n pixels, when the grey level of each of the pixels is 
inspected, much time will be required for the Inspection. Usually, 
n » about 512 and n x n = 262144. 
S On the other hand, in this embodiment, in Step 1, a 

"quad-tree" is made and, in Step 2, in selecting a dark region of a 
grey level of a value not larger than a predetermined value, a 
macro-inspection to a micro-inspection are made in the order of 
picture-images of less pixels by using the above-mentioned 
10 "quad-tree". Therefore, the processing time can be remarkably 
reduced* 

The present invention is not limited to the above-mentioned 
embodiments. For example, Step 3 is not always required. 

The endoscope may be inserted manually by the curvature 

15 operation and advancing insertion operation by the endoscope 

operator, following the endoscope insertion direction detected by the 
method of the present invention or it may be inserted by 
automatically directing the sheath tip in the detected inserting 
direction by the apparatus. 

20 As explained above, according to the present invention, 

there is an effect that the endoscope insertion direction can 'be 
detected simply within a short processing time. 



CLAIMS: 

1. A method of detecting, the direction of insertion of an 
endoscope sheath comprising the steps of: 

forming an endoscope display image, and providing means to 
5 construct therefrom a plurality of picture-images differing in the 
number of pixels (or picture elements); 

selectively locating a dark region in a picture-image of a 
predetermined number of pixels by inspecting the picture-images 
sequentially in the order of less pixels to establish the grey levels 
10 of the respective pixels in the plurality of picture-images thus 
formed; and 

finally utilising the said dark region located by the said 
selective step as the endoscope sheath insertion direction at that 
given instant. 

15 

2. A method as claimed In Claim 1, wherein the said forming 
step includes gradually forming picture-images of fewer pixels, in 
each of which the number of pixels is reduced by a quarter so that, 
for each macro-pixel formed by four subsidiary pixels of the previous 

20 picture-image, the grey level of one macro-pixel in the newly formed 
picture-image of fever pixels will have an average value of tlje grey 
levels of the four subsidiary pixels arranged 2 x 2 in the previous 
picture-image of more pixels that have been combined to form said one 
macro-pixel. 
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3. A method as claimed in Claim 2, wherein the said selective 
step includes inspecting the grey levels of the respective pixels in 
a picture-Image, selecting that macro-pixel nearest to a required 
grey level, then inspecting the grey levels of the four subs ida iry 
pixels of that selected macro-pixel and selecting the subsidiary 
pixel nearest to the required grey level. 

4. A method as claimed in Claim 3, wherein the said selective 
step includes parallel processing using a plurality of electronic 
computers. 

5. A method as claimed in Claim 3, wherein the said selective 
step includes means ending the Inspection when the differences of the 
grey levels of the said four subsidiary pixels to that of their 
macro-pixel is found to be not larger than a predetermined value, 

6. A method as claimed in Claim 3, wherein the said selective 
step includes means ending the inspection when the ratios of the grey 
levels of the said four subsidiary pixels to that of their 
macro-pixel is found to be not larger than a predetermined value. 

7. a method as claimed in any preceding Claim, wherein the said 
forming step includes means forming a plurality of picture-images in 
which pixels are considered to be white In any black region lying 
outside the endoscope display image. 

8. A method as claimed in any preceding Claim, wherein the said 
forming step includes means forming a plurality of picture-images 
removing any black region lying outside the endoscope display Image. 
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9. A method as claimed in any preceding Claim, wherein the said 
selective step includes means setting the upper limit and lower limit 
of the "floor" at which the inspection may be concluded in accordance 
with a required precision. 

5 

10. A method as claimed in any preceding Claim, comprising a 
further step of selecting a region within a predetermined grey level 
range near the said dark region selected in the said selective step, 
and combining the said region with that said dark region. 

10 

11. A method as claimed in any preceding Claim, wherein the grey 
level of each said pixel is indicated by a digital amount in the said 
forming step and the said selective step. 

15 12. A method as claimed in any preceding Claim, wherein the 

endoscope display picture-image is obtained by a television camera 
fitted to an eye-piece of an endoscope, whereby naked-eye observation 
is possible. 

20 13. A method as claimed in any preceding Claim, wherein the 

endoscope display image is obtained from an imaging means provided in 
the endoscope. 

14. A method as claimed in any preceding Claim, in which said 

25 dark region selectively located is considered to be the endoscope 
inserting direction following computer analysis thereof. 
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15. A method of detecting the direction of insertion of an 

endoscope sheath by computer analysis, substantially as described 
with reference to one or more of Figures 1 to 20 . 
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